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(54) Picture processing method and apparatus 



(57) It is determined whether or not an input video 
signal is a side panel signal, a letter box signal, or a full 
line signal. If the input video signal is a side panel signal 
or a letter box signal, when a multiple-picture displaying 
process or a reduced picture displaying process is per- 
formed, only a signal of an effective picture area is ex- 
tracted. With only the signal of the effective picture area, 



the picture sizes are adjusted and the pictures are com- 
bined. Thus, even if an input video signal is a side panel 
signal or a letter box signal, the picture does not become 
small. In addition, when the reduced picture displaying 
process is performed, since a wasteful picture portion is 
not processed, the utilization eff iciency of the screen is 
improved. 
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Description 

[0001] The present invention relates to a picture 
processing apparatus and a picture processing method. 
[0002] Fig. 1 shows an example of a displaying appa- 
ratus of a television receiver that can perform both a 
multiple-picture display of which a plurality of pictures 
are displayed in a row on a background screen and a 
reduced picture display of which a reduced picture is dis- 
played along with an on-screen picture such as text. 
[0003] In Fig. 1 , a plurality of video signals are input 
from inputterminals 102A, 102B, and so forth to an input 
signal selecting portion 101. The input signal selecting 
portion 101 selects two video signals of two sources that 
are displayed on the screen from the video signals that 
are input from the input terminals 102A, 102B, and so 
forth. 

[0004] The selected two video signals of two sources 
are supplied to A/D converters 103A and 103B. The A/ 
D converters 103A and 103B digitize two video signals 
of two sources that have been selected by the input sig- 
nal selecting portion 101 . The two video signals of two 
sources that have been digitized by the A/D converters 
1 03A and 1 03B are supplied to a picture processing por- 
tion 104. 

[0005] The picture processing portion 1 04 performs a 
multiple-picture displaying process and a reduced pic- 
ture displaying process. To accomplish such processes, 
the picture processing portion 1 04 has a picture memory 
105. 

[0006] When two pictures are displayed, the two video 
signals of two sources are written to the picture memory 
105. At desired timings, the two video signals of two 
sources are read from the picture memory 1 05 and dis- 
played on the background screen. At that point, the two 
video signals of two sources are thinned out or interpo- 
lated so as to adjust the picture sizes. 
[0007] Output signals of the picture processing por- 
tion 104 are supplied to a D/A converter 106. The D/A 
converter 1 06 converts the output signals of the picture 
processing portion 104 into analog signals. Output sig- 
nals of the D/A converter 1 06 are supplied to a C RT dis- 
play 107. 

[0008] Video signals supplied to the input terminals 
1 02A, 1 02B, and so forth may for example letter box sig- 
nals or side pane! signals that contain non-picture por- 
tions. 

[0009] There are three types of formats of video sig- 
nals. The three types of video signals differ in the 
number of effective pixels. The number of effective pix- 
els of the first type is (720 x 480). The number of effec- 
tive pixels of the second type is (1280 x 720). The 
number of effective pixels of the third type is (1920 x 
1080). 

[001 0] In the first format of which the number of effec- 
tive pixels is (720 x 480), the angle of view is (4 : 3) that 
is a standard screen. In the second format of which the 
number of effective pixels is (1280 x 720) and the third 



format of which the number of effective pixels is (1920 
x 1 080), the angle of view is (1 6 : 9) that is a wide screen. 
[001 1 ] As shown in Fig. 2A, when the angle of view is 
16:9 and the aspect ratio of the effective picture area 

5 is 4 : 3, black non-picture portions are added to the left 
and right of the pictu re so that the angle of view becomes 
16:9. Such a signal is referred to as side panel signal. 
[0012] In addition, as shown in Fig. 2B, when the an- 
gle of view is 4 : 3 and the aspect ratio of the effective 

10 picture area is 16 :9, black non-picture portions are add- 
ed to the top and bottom of the picture so that the angle 
of view becomes 4 : 3. Such a signal is referred to as 
letter box signal. 

[0013] Thus, as shown in Fig. 2A, a side panel signal 

'5 has non-picture portions on the left and right of the pic- 
ture. On the other hand, as shown in Fig. 2B, a letter 
box signal has non-picture portions at the top and bot- 
tom of the picture. Thus, as was described above, when 
an input video signal is written to the picture memory 

20 1 05 and read therefrom at a proper timing correspond- 
ing to the screen position in a predetermined picture 
size, if the input video signal is a letter box signal or a 
side panel signal, the non-picture portions are also proc- 
essed. Thus, when a multiple-picture displaying process 

25 is performed, the picture of the side panel signal or letter 
box signal becomes small. On the other hand, when the 
reduced picture displaying process is performed, the 
display area on the screen cannot be effectively used. 
[0014] Figs. 3A, 3B, and 3C show an example of 

30 which two pictures are displayed with a picture of a side 
panel signal and a picture of a full line signal. 
[0015] As shown in Fig. 3A, all the picture of a side 
panel signal S1 01 that contains left and right non-picture 
portions is written to the picture memory. The picture 

35 size of the video signal written to the picture memory is 
adjusted in the size of all the picture. On the other hand, 
as shown in Fig. 3B, a full line signal S102 is written to 
the picture memory. The picture size of all the picture of 
the video signal is adjusted in the picture memory. The 

40 adjusted picture is read from the picture memory and 
displayed on the background screen. As a result, as 
shown in Fig. 3C, two pictures that are a picture G101 
formed with the side panel signal S101 and a picture 
G1 02 formed with the full line signal S1 02 are displayed. 

45 [0016] As shown in Figs. 3A and 3B, the aspect ratio 
of the effective picture area of each of the two signals 
S101 and S102 is (4 : 3). Thus, the pictures will be dis- 
played in the same size. However, as shown in Fig. 3C, 
when the two pictures G101 and G102 are displayed in 

50 a row, due to the left and right non-picture portions, the 
picture G1 01 formed with the side panel signal S1 01 be- 
comes smaller than the picture G102 formed with the 
full line signal S1 02. 

[0017] Figs. 4A and 4B show an example of which the 
55 area of a picture of a side panel signal S111 is reduced 
on the display screen. As shown in Fig. 4A, all the picture 
of the side panel signal S111 that contains the left and 
right non-picture portions is written to the picture mem- 
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ory. The picture size of the video signal written to the 
picture memory is adjusted in the size of all the picture. 
The adjusted picture is read from the picture memory 
and displayed on the background screen. As shown in 
Fig. 4B, in addition to a picture Gill formed with the side 
panel signal S111 , an on-screen picture G112 such as 
text is displayed. 

[001 8] In this case, since the picture of the side panel 
signal S111 contains the left and right non-picture por- 
tions, The reduced picture G1 1 1 has a wasteful portion. 
Due to the wasteful portion, the area for the on-screen 
display picture G112 such as text becomes small. Con- 
sequently, the utilization efficiency of the screen be- 
comes low. 

[001 9] Various respective aspects and features of the 
invention are defined in the appended claims. Features 
from the dependent claims may be combined with fea- 
tures of the independent claims as appropriate and not 
merely as explicitly set out in the claims. 
[0020] Embodiments of the invention are suitable for 
a television receiver that performs both a multiple-pic- 
ture display of which a plurality of pictures are displayed 
in a row on a background screen and a reduced picture 
display of which a reduced picture is displayed along 
with an on-screen picture such as text. 
[0021] Embodiments of the present invention can pro- 
vide a picture displaying apparatus and a picture dis- 
playing method that allow pictures to be prevented from 
becoming small in a multiple-picture displaying process 
and a wasteful picture portion to be prevented from be- 
ing processed without deterioration of utilization factor 
of the screen in a reduced picture displaying process 
even if input video signals are side panel signals or letter 
box signals. 

[0022] A first aspect of the present invention is a pic- 
ture processing apparatus, comprising a determining 
means for determining whether or not an input video sig- 
nal is a signal of which a non-picture portion is added to 
the periphery of an effective picture area, and a picture 
processing means for extracting a signal of the effective 
picture area from the input video signal, adjusting the 
picture size using the signal of the effective picture area, 
and combining the picture when the determined result 
of the determining means represents that the input video 
signal is a signal of which a non-picture portion is added 
to the periphery of the effective picture area. 
[0023] A second aspect of the present invention is a 
picture processing method, comprising the steps of (a) 
determining whether or not an input video signal is a sig- 
nal of which a non-picture portion is added to the periph- 
ery of an effective picture area, and (b) extracting a sig- 
nal of the effective picture area from the input video sig- 
nal, adjusting the picture size using the signal of the ef- 
fective picture area, and combining the picture when the 
determined result at step (a) represents that the input 
video signal is a signal of which a non-picture portion is 
added to the periphery of the effective picture area. 
[0024] It is determined whether or not an input video 



signal is a video signal that contains non-picture por- 
tions such as a side panel signal or a letter box signal. 
When a multiple-picture displaying process or a reduced 
picture displaying process is performed, if the input vid- 

5 eo signal is a side panel signal or a letter box signal, 
only a video signal of the effective picture area is ex- 
tracted and read from a picture memory and displayed 
on the screen. Thus, when the multiple-picture display- 
ing process is performed, even if an input video signal 

10 is a side panel signal or a letter box signal, the display 
picture does not become small. On the other hand, when 
the reduced picture displaying process is performed, the 
display area can be effectively used. 
[0025] The invention will now be described by way of 

15 example with reference to the accompanying drawings, 
throughout which like parts are referred to by like refer- 
ences, and in which: 

Fig. 1 is a block diagram showing an example of a 
20 conventional picture displaying apparatus; 

Figs. 2A and 2B are schematic diagrams showing 
a display example of the conventional picture dis- 
playing apparatus; 

Figs. 3A, 3B, and 3C are schematic diagrams show- 
25 jng a display example of the conventional picture 
displaying apparatus; 

Figs. 4A and 4B are schematic diagrams showing 
a display example of the conventional picture dis- 
playing apparatus; 
30 Fig. 5 is a block diagram showing an embodiment 
of the present invention; 

Figs. 6A and 6B are block diagrams for explaining 
a display example of the embodiment of the present 
invention; 

35 Figs. 7A, 7B, and 7C are schematic diagrams show- 
ing a display example according to the embodiment 
of the present invention; 

Figs. 8A and 8B are schematic diagrams showing 
a display example according to the embodiment of 
40 the present invention; 

Figs. 9A and 9B are schematic diagrams showing 
a display example according to the embodiment of 
the present invention; 

Figs. 10A, 10B, 10C, and 10D are schematic dia- 
45 grams showing a display example according to the 
embodiment of the present invention; 
Fig. 11 is a block diagram showing another embod- 
iment of the present invention; 
Figs. 12A, 12B, 12C, and 12D are schematic dia- 
50 grams showing a display example of the other em- 
bodiment of the present invention; 
Figs. 13A, 13B, 13C, and 13D are schematic dia- 
grams showing a display example according to the 
other embodiment of the present invention; 
55 Figs. 1 4A and 1 4B are schematic diagrams showing 
a display example according to the other embodi- 
ment of the present invention; and 
Figs. 1 5A and 1 5B are schematic diagrams showing 
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a display example according to the other embodi- 
ment of the present invention. 

[0026] Next, with reference to the accompanying 
drawings, embodiments of the present invention will be 
described. Fig. 5 shows an example of a picture display- 
ing apparatus according to the present invention. 
[0027] In Fig. 5, reference numeral 1 is an interface 
that inputs a plurality of video signals. 
[0028] There are the following picture formats for vid- 
eo signals. The first picture format is (720 x 480) format 
of which the number of effective pixels is 720 x 480. The 
second picture format is (1280 x 720) format of which 
the number of effective pixels is 1280 x 720. The third 
picture format is (1920 x 1080) format of which the 
number of effective pixels is 1920 x 1080. In the (720 x 
480) format, the angle of view is (4 : 3) that is a standard 
screen. In the (1 280 x 720) format and the (1 920 x 1 080) 
format, the angle of view is (1 6 : 9) that is a wide screen. 
In addition, in each of the (720 x 480) format, the (1 280 
x 720) format, and the (1920 x 1080) format, there are 
two types of scanning formats that are interlace scan- 
ning and progressing scanning. 
[0029] As the interface 1 , S terminal, D terminal, com- 
ponent terminal, or the like is used. 
[0030] The D terminal was standardized for a digital 
broadcast. In the D terminal, (Y, Pb, and Pr) component 
signals and a picture format identifying signal are trans- 
mitted. The D terminal has five types D1 to D5 that de- 
pend on broadcast formats. 

[0031] The S terminal is used to separately input a lu- 
minance signal and a chroma signal. The component 
terminal uses three pins for component signals. A stand- 
ard television (480i) uses the S terminal, whereas a 
HDTV (1080i) uses the component terminal having 
three pins. 

[0032] The interface 1 has a connection ID detecting 
portion 4. The connection ID detecting portion 4 detects 
the picture format of a video signal that is input. 
[0033] When a video signal is input using the D termi- 
nal or the S terminal, the picture format of the input video 
signal can be determined with an output signal of the 
interface 1 . An outputsignal of the connection ID detect- 
ing portion 4 is supplied to a microprocessor 5. The mi- 
croprocessors determines the picture format of an input 
video signal with an output signal of the connection ID 
detecting portion 4. 

[0034] Video signals that are input from the interface 
1 are supplied to an input signal selecting portion 2. The 
input signal selecting portion 2 selects two video signals 
of two sources that are displayed on the screen from the 
video signals that are input through the interface 1 . 
[0035] The two video signals of two sources that are 
selected by the input signal selecting portion 2 are sup- 
plied to A/D converters 6A and 6B. The A/D converters 
6A and 6B digitize the two video signals of two sources 
selected by the input signal selecting portion 2. Output 
signals of the A/D converters 6A and 6B are supplied to 



a picture processing portion 7. 
[0036] The two video signals of two sources selected 
by the input signal selecting portion 2 are supplied to an 
additional information detecting portion 8. The addition- 

s al information detecting portion 8 detects additional in- 
formation added to or superimposed with the signals 
and determines the picture formats of the input video 
signals corresponding to the detected additional sig- 
nals. For example, in an EDTV-II ID signal or an ID-1 

10 signal, the aspect ratio of a picture and the angle-of-view 
information are superimposed as additional information. 
The additional information detecting portion 8 detects 
additional information added to or superimposed with an 
input video signal. The additional information is supplied 

is to the microprocessor 5. The microprocessor 5 deter- 
mines the picture format of an input video signal with 
additional information detected by the additional infor- 
mation detecting portion 8. 

[0037] With an output signal of the connection ID de- 
20 tecting portion 4, corresponding to the information of the 
interface 1 , the picture format of an input video signal is 
determined. In addition, the additional information de- 
tecting portion 8 determines the picture format of an in- 
put video signal with additional information added there- 
25 to or superimposed therewith. Corresponding to the de- 
termined picture format, it is determined whether the in- 
put video signal is a side panel signal, a letterbox signal, 
or a full line signal. 

[0038] In addition, the two video signals of two sourc- 

30 es selected by the input signal selecting portion 2 are 
supplied to a non-signal detecting portion 9. The non- 
signal detecting portion 9 compares the level of each 
input video signal with a predetermined level. Output 
signals of the non-signal detecting portion 9 are supplied 

35 to the microprocessor 5. The microprocessor 5 detects 
the size of the effective picture area of each input video 
signal with each compared output. When the effective 
picture area in the horizontal direction is small, the mi- 
croprocessor 5 determines that the input video signal is 

40 a side panel signal. When the effective picture area in 
the vertical direction is small, the microprocessor 5 de- 
termines that the input video signal is a letter box signal. 
[0039] As will be described later, information that rep- 
resents whether an input video signal is a letter box sig- 

45 nal or a side panel signal is used for a multiple-picture 
displaying process and a reduced picture displaying 
process. 

[0040] The picture processing portion 7 performs a 
process for placing pictures corresponding to the two 

so video signals of two sources selected by the input signal 
selecting portion 2 at proper positions on the back- 
ground screen. In addition, the picture processing por- 
tion 7 performs a process for displaying a picture corre- 
sponding to one input video signal as a reduced picture 

55 on the background screen and for displaying an on- 
screen picture such as text on the background screen. 
[0041 ] An output signal of the picture processing por- 
tion 7 is supplied to a D/A converter 12. The D/A con- 
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verter 12 converts the output signal of the picture 
processing portion 7 into an analog signal. An output 
signal of the D/A converter 1 2 (namely, an analog signal) 
is supplied to a CRT display 1 3. 
[0042] As described above, the picture processing 
portion 7 performs the multiple-picture displaying proc- 
ess and the reduced picture displaying process. These 
processes are accomplished by writing input video sig- 
nals to a picture memory 1 1 , thinning out or interpolating 
video signals therein at proper timings, and reading the 
resultant video signals so as to adjust the picture sizes. 
[0043] For example, when the multiple-picture dis- 
playing process is performed, two video signals of two 
sources selected by the input signal selecting portion 2 
are written to the picture memory 1 1 . The two video sig- 
nals of two sources are thinned out or interpolated at 
proper timings so that desired picture sizes are obtained 
corresponding to the display positions on the screen. 
The resultant pictures are read from the picture memory 
11 and combined on the background screen. 
[0044] However, each input video signal may be a 
side panel signal or a letter box signal. In the side panel 
signal, the aspect ratio of the effective picture area is 
(4 : 3) and black non-picture portions are added to the 
left and right thereof so that the angle of view becomes 
16 : 9. In the letter box signal, the aspect ratio of the 
effective picture area is (16 : 9) and black non-picture 
portions are added to the top and bottom thereof so that 
the angle of view becomes 4 : 3. Since a side panel sig- 
nal and a letter box signal contain non-picture portions, 
when an input video signal is written to the picture mem- 
ory 11, adjusted in a predetermined picture size at a 
proper timing corresponding to a position on the screen, 
and read from the picture memory 11 , if the input video 
signal is a side panel signal or a letter box signal, a dis- 
playing process is performed for the non-picture por- 
tions. 

[0045] Thus, according to the embodiment of the 
present invention, the microprocessor 5 determines 
whether each of the selected video signals is a side pan- 
el signal, a letter box signal, or a full line signal. In the 
case that each of the selected video signals is a side 
panel signal or a letter box signal, when the multiple- 
picture displaying process or the reduced picture dis- 
playing process is performed, only a video signal of the 
effective picture area is extracted from the video signal 
stored in the picture memory 11 . 
[0046] In other words, as shown in Figs. 6A and 6B, 
the picture memory 11 has a memory space composed 
of horizontal addresses xa to xb and vertical addresses 
ya to yb for one screen of an input video signal. Thus, 
an input video signal for one screen is written to the 
memory space composed of horizontal addresses xa to 
xb and vertical addresses ya to yb. 
[0047] When an input video signal is a full line signal, 
all the video signal written to the memory space com- 
posed of horizontal addresses xa to xb and vertical ad- 
dresses ya to yb is effective. 



[0048] On the other hand, when an input video signal 
is a side panel signal, as shown in Fig. 6A, the portion 
surrounded by horizontal addresses xc to xd and vertical 
addresses ya to yb of the video signal of one screen 

5 written to the picture memory 11 is an effective picture 
area. Thus, the portion surrounded by horizontal ad- 
dresses xa to xc and vertical addresses ya to yb and the 
portion surrounded by horizontal addresses xd to xb and 
vertical addresses ya to yb are non-picture portions. 

10 [0049] On the other hand, when an input video signal 
is a letter box signal, as shown in Fig. 6B, the portion 
surrounded by horizontal addresses xa to xb and verti- 
cal addresses yc to yd of the video signal for one screen 
written to the picture memory 11 is an effective picture 

15 area. The portion surrounded by horizontal addresses 
xa to xb and vertical addresses ya to yc and the portion 
surrounded by horizontal addresses xa to xb and verti- 
cal addresses yd to yb are non-picture portions. 
[0050] When the multiple-picture displaying process 

20 or the reduced picture displaying process is performed, 
if an input video signal is a side panel signal, as shown 
in Fig. 6A, only a video signal of the effective picture 
area surrounded by horizontal addresses xc to xd and 
vertical addresses ya to yb is extracted from the video 

25 signal for one screen written to the picture memory 11 
and displayed. On the other hand, if an input video signal 
is a letter box signal, as shown in Fig. 6B, only a video 
signal of the effective picture area surrounded by hori- 
zontal addresses xa to xb and vertical addresses yc to 

30 yd is extracted from the input video signal and displayed. 
[0051] Thus, when the multiple-picture displaying 
process or the reduced picture displaying process is 
performed, if an input video signal is a side panel signal 
or a letter box signal, only a video signal of the effective 

35 picture signal is extracted from the input video signal 
and displayed . As a result, when the multiple-picture dis- 
playing process is performed, a picture formed with a 
side panel signal or a letterbox signal can be prevented 
from becoming small. When the reduced picture dis- 

40 playing process is performed, a wasteful portion can be 
prevented from being displayed. Thus, the screen can 
be effectively used. 

[0052] As was described above, the determination of 
which an input video signal is a side panel signal, a letter 
45 box signal, or a full line signal can be performed with an 
output signal of the connection ID detecting portion 4, 
an output signal of the additional information detecting 
portion 8, or an output signal of the non-signal detecting 
portion 9. 

so [0053] In other words, when the interface 1 is S ter- 
minal, D terminal, or the like, the angle-of-view informa- 
tion and aspect ratio of an input video signal can be ob- 
tained with information of the interface 1 . On the other 
hand, when an input video signal is a signal of which 

55 additional signal is added to or superimposed with a vid- 
eo signal for example an EDTV-II ID signal or an ID-1 
signal, the aspect ratio and angle-of-view information 
can be detected from the additional information, lnfor- 
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mation of the interface 1 is detected by the connection 
ID detecting portion 4. The detected information is sup- 
plied to the microprocessors. The additional information 
added to or superimposed with the input video signal is 
detected by the additional information detecting portion 
8 and supplied to the microprocessor 5 
[0054] The microprocessor 5 obtains the angle-of- 
view information and aspect ratio information of the in- 
put video signal from the information detected by the 
connection ID detecting portion 4 or the information de- 
tected by the additional information detecting portion 8 
and determines whether the input video signal is a side 
panel signal, a letter box signal, or a full line signal. 
[0055] When the determined results with the informa- 
tion of angle-of-view and the information of aspect ratio 
represent that the angle of view is (4 : 3) and the aspect 
ratio of the effective picture area is (16 : 9), the input 
video signal is a letter box signal of which non-picture 
portions are added to the top and bottom of a picture 
whose effective picture area is (1 6 : 9) so that the angle 
of view becomes (4:3). Likewise, the determined result 
with information of the aspect ratio represents that the 
aspect ratio of the effective picture area is (4 : 3), the 
input video signal is a side panel signal of which non- 
picture portions are added to the left and right of a pic- 
ture whose effective area is (4 : 3) so that the angle of 
view becomes (16 : 9). 

[0056] However, there is a situation of which informa- 
tion about the picture format of an input video signal can- 
not be obtained with neither information of the interface 
1 , nor additional information of the input video signal. In 
this case, the non-signal detecting portion 9 compares 
the level of an input video signal with a predetermined 
level. With the compared signal, the size of the effective 
picture area is detected. When the effective picture area 
in the horizontal direction is small, it is determined that 
the input video signal is a side panel signal. On the other 
hand, when the effective picture area in the vertical di- 
rection is small, it is determined that the input video sig- 
nal is a letter box signal. 

[0057] In the example, with three types of information 
that are information of the interface 1 , information added 
to or superimposed with an input video signal, and de- 
tected information of non-picture portions of an input vid- 
eo signal, it is determined whether the input video signal 
is a side panel signal, a letter box signal, or a full line 
signal. Of course, it is not necessary to use all three 
types of information. Instead, either information of the 
interface 1 , information added to or superimposed with 
an input video signal, or detected information of non- 
picture portions of the input video signal may be used. 
The combination and priority of these three types of in- 
formation will be determined in consideration of the in- 
stallation environment, cost, performance, and so forth. 
[0058] When the multiple-picture displaying process 
or the reduced picture displaying process is performed, 
if an input video signal is a side panel signal or a letter 
box signal, only a video signal of the effective picture 



area is extracted from the input video signal and dis- 
played. Thus, when the input video signal is a side panel 
signal or a letter box signal, the size of the picture does 
not become small. In addition, the display area of the 

5 screen can be effectively used. 

[0059] Figs. 7A, 7B, and 7C show an example of a 
multiple-picture (two picture) display. In this example, it 
is assumed that one video signal of two input video sig- 
nals of two sources is a side panel signal S1 and the 

10 other video signal thereof is a full line signal S2. 

[0060] As shown in Fig. 7A, all the picture of the side 
panel signal S1 that contains left and right non-picture 
portions is written to the picture memory. A video signal 
of the effective picture area is extracted from the video 

15 signal. The picture size of the effective picture area is 
adjusted in the picture memory. The adjusted picture is 
read from the picture memory and displayed. On the oth- 
er hand, as shown in Fig. 7B, ail the picture of the full 
line signal S2 is written to the picture memory. In the 

20 picture memory, the picture size is adjusted to the size 
of all the picture. The adjusted picture is read from the 
picture memory. The picture is displayed on the back- 
ground screen. Thus, as shown in Fig. 7C, two pictures 
of a picture G1 formed with the side panel signal S1 and 

25 a picture G2 formed with the full line signal S2 are dis- 
played. 

[0061] As shown in Fig. 7C, in the example, when only 
the effective picture area is extracted from the side panel 
signal S1 , the size of the picture G1 formed with the side 

30 panel signal S1 can be matched with the size of the pic- 
ture G2 formed with the full line signal S2. 
[0062] Figs. 8A and 8B show a display example of a 
reduced picture in the case that an input video signal is 
a side panel signal. As shown in Fig. 4A, all the picture 

35 of the side panel signal S11 that contains left and right 
non-picture portions is written to the picture memory. A 
video signal of the effective picture area is extracted 
from the video signal written to the picture memory. The 
picture size is adjusted in the effective picture area. The 

40 adjusted picture is read from the picture memory. The 
picture is displayed on the background picture. As 
shown in Fig. 8B, in addition to a picture G11 formed 
with the side panel signal S1 1 , an on-screen picture G1 2 
such as text is displayed. 

45 [0063] As shown in Fig. 8B, when a reduced picture 
is displayed, if an input video signal is a side panel sig- 
nal, a video signal of the effective picture area is extract- 
ed from the video signal written to the picture memory 
and displayed. Thus, non-picture portions in the hori- 

50 zontal direction are removed from the a reduced picture 
G1 1 formed with the side panel signal S 1 1 . Consequent- 
ly, the on-screen picture G12 such as text can be effec- 
tively displayed. 

[0064] Figs. 9A and 9B show an example of a reduced 
55 picture display in the case that an input video signal is 
a letter box signal. As shown in Fig. 9A, all the picture 
of a letter box signal S21 that contains top and bottom 
non-picture portions is written to the picture memory. A 
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video signal of the effective picture area is extracted 
from the input video signal written to the picture memory. 
In the effective picture area, the picture size is adjusted. 
The adjusted picture is read from the picture memory 
and displayed on the background screen. As shown in 
Fig. 9B, in addition to a picture G21 formed with the side 
panel signal S21 , an on-screen picture S22 such as text 
is displayed. 

[0065] As shown in Fig. 9B, when a reduced picture 
is displayed, if an input video signal is a letter box signal 
S21 , a video signal of the effective picture area is ex- 
tracted from the input video signal written to the picture 
memory and displayed. Thus, the reduced picture G21 
formed with the letter box signal S21 does not contain 
non-picture portions in the vertical direction. Thus, the 
on-screen picture G22 such as text can be effectively 
displayed. 

[0066] In the above description, as an example of the 
multiple-picture displaying process, two pictures are dis- 
played. Of course, more than two pictures can be dis- 
played. 

[0067] Figs. 10A and 10B show an example of three 
pictures displayed as a multiple-picture display. In the 
example, as shown in Fig. 10A, all the picture of a side 
panel signal S31 that contains left and right non-picture 
portions is written to the picture memory. A video signal 
of the effective picture area is extracted from the video 
signal written to the picture memory. In the effective pic- 
ture area, the picture size is adjusted. The adjusted pic- 
ture is read from the picture memory. 
[0068] All the picture of a letter box signal that con- 
tains top and bottom non-picture portions is written to 
the picture memory. A video signal of the effective pic- 
ture area is extracted from the video signal written to the 
picture memory. In the effective picture area, the picture 
size is adjusted. The adjusted picture is read from the 
picture memory and displayed. 

[0069] All the picture of a letter box signal S33 that 
contains top and bottom non-picture portions is written 
to the picture memory. A video signal of the effective 
picture area is extracted from the video signal written to 
the picture memory. In the effective picture area, the pic- 
ture size is adjusted. The adjusted picture is read from 
the picture memory and displayed. 
[0070] Thus, as shown in Fig. 10D, three pictures of 
a picture G31 formed with the side panel signal S31 , a 
picture G32 formed with the letter box signal S32, and 
a picture G33 formed with the letter box signal S33 are 
displayed in a row on the background screen. 
[0071] In the above description, all the picture of a vid- 
eo signal is pre-written to the picture memory 11 . When 
the multiple-picture displaying process or the reduced 
picture displaying process is preformed, if an input video 
signal is a side pane) signal or a letter box signal, only 
a video signal of the effective picture area is read and 
extracted from the input picture signal. However, when 
an input video signal is a side panel signal or a letter 
box signal, only a video signal of the effective picture 



area may be written to the picture memory. The input 
video signal is read from the picture memory so as to 
extract a video signal of the effective picture area from 
the input video signal. 

5 [0072] Fig. 11 shows another embodiment of the 
present invention. In the example, in addition to two vid- 
eo signals of two sources selected by the input signal 
selecting portion 2, a received picture of a digital satellite 
broadcast corresponding to MPEG (Moving Picture 

10 Coding Experts Group) 2 standard can be displayed. 
For simplicity, in Fig. 11 , similar portions to those in Fig. 
5 will be denoted by similar reference numerals and their 
description will be omitted. 

[0073] In Fig. 11 , an intermediate frequency signal of 

is a digital satellite broadcast is supplied to an input termi- 
nal 20. For example, in a digital BS (Broadcast Satellite) 
broadcast, a radio wave of 12 GHz band transmitted 
from a satellite is received by a parabola antenna (not 
shown). An LNB (Low Noise Block down converter) dis- 

20 posed on the parabola antenna converts the radio wave 
into a BS broadcast intermediate frequency signal BS- 
1 F.The intermediate frequency signal BS-1 F is supplied 
to the input terminal 20. In a digital CS (Communication 
Satellite) broadcast, the LNB converts the radio wave 

25 into a CS broadcast intermediate frequency signal 
CS-IF. The CS broadcast intermediate frequency signal 
CS-IF is supplied to the input terminal 20. 
[0074] The signal that is input from the input terminal 
20 is supplied to a front end 21 . The front end 21 de- 

30 modulates the received signal and outputs an MPEG2 
transport stream. The transport stream demodulated by 
the front end 21 is supplied to a demultiplexer 22. The 
demultiplexer 22 depacketizs the transport stream cor- 
responding to PID. An output signal of the demultiplexer 

35 22 is supplied to an MPEG2 video decoder 24. 

[0075] The MPEG2 video decoder 24 decompresses 
the MPEG2 video signal. The video signal decoded by 
the MPEG2 video decoder 24 is supplied to a picture 
processing portion 7. 

40 [0076] In the example, an input terminal 23 is dis- 
posed so as to decode an MPEG2 transport stream of 
another medium received from the outside of the appa- 
ratus. In addition, an input terminal 25 is disposed so as 
to decode an MPEG2 video signal of another medium 

45 received from the outside of the apparatus. 

[0077] Thus, in the example, an MPEG2 transport 
stream is decoded. The decoded video signal is sent to 
the picture processing portion 7. 
[0078] In addition to video signals selected by the in- 

50 put signal selecting portion 2, the picture processing 
portion 7 performs a multiple-picture displaying process 
or a reduced picture displaying process for the video sig- 
nal decoded by the MPEG2 video decoder 24. 
[0079] As was described above, when the multiple- 

55 picture displaying process or the reduced picture dis- 
playing process is performed, if each input video signal 
is a side panel signal or a letter box signal, a process 
for extracting only a video signal of the effective picture 
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area from the input video signal and displaying the ef- 
fective picture area is performed. 
[0080] In the above-described example, the determi- 
nation of whether an input video signal is a side panel 
signal, a letter box signal, or a full line signal is per- 
formed with information of the interface 1, information 
added to or interpolated with the input video signal, or 
detected information of non-picture portions of the input 
video signal. On the other hand, when an input video 
signal is an MPEG2 decode signal, the determination of 
whether an input video signal is a side panel signal or a 
letter box signal is performed with information transmit- 
ted as an MPEG2 transport stream. 
[0081] In other words, the aspect ratio and angle of 
view of a picture transmitted in an MPEG2 system can 
be represented with SequenceJHeader . and 
Sequence_Display_Extension. 

[0082] For example, in a BS digital broadcast system, 
Horizontal_Size_Value (HSV) of SequenceJHeader 
represents the horizontal size of the entire picture. 
Vertical_Size_Value (VSV) represents the vertical size 
of the entire picture. Aspect_RatioJnformation (ARI) 
represents the aspect ratio of the effective picture area. 
[0083] DisplayJHorizontaLSize (DHS) of 
Sequence_Display_Extension represents the horizon- 
tal size of the effective picture area. 
Display_Vertical_Size (DVS) represents the vertical 
size of the effective picture area. 
[0084] Thus, when an input video signal is a full line 
signal, HorizontaLSizeJ/alue (HSV) matches 
DisplayJHorizontaLSize (DHS). In addition, 
Value_Size_Value (VSV) matches 

Display_VerticaLSize (DVS). 

[0085] When an input video signal is a letter box sig- 
nal, Horizontal_Size_Value (HSV) matches 
DisplayJHorizontaLSize (DHS). Display_VerticaLSize 
(DVS) is smaller than VerticaLSize_Vaiue (VSV). 
[0086] When an input video signal is a side panel sig- 
nal, DisplayJHorizontaLSize (DHS) is smaller than 
Ho1izontal_Size_Value (HSV). Vertica1_Size_Value 
(VSV) matches Display_Vertical_Size (DVS). 
[0087] In such a manner, the value of 
Horizontal_Size_Value (HSV) is compared with the val- 
ue of DisplayJHorizontaLSize (DHS). In addition, the 
value of Vertical_Size_Value (VSV) is compared with 
the value of Display__VerticaLSize (DVS). With the com- 
pared results, it can be determined whether an input vid- 
eo signal is a full line signal, a side panel signal, or a 
letterbox signal. In addition, with Horizontal_Size (DHS) 
and Display_Vertical_Size (DVS), the size of the effec- 
tive picture area can be determined. 
[0088] Thus, when an input video signal is an MPEG2 
video signal, Horizontal_Size_Value (HSV) is compared 
with DisplayJHorizontaLSize (DHS). In addition, 
Vertical_Size_Value (VSV) is compared with 
Display_Vertical_Size (DVS). A process for extracting a 
video signal of the effective picture area from the input 
video signal corresponding to the smaller parameters. 



[0089] Next, a display example of which video signals 
selected by the input signal selecting portion 2 and a 
video signal decoded by the MPEG2 video decoder 24 
are supplied to the picture processing portion 7 will be 

5 described. 

[0090] Figs. 12A to 12D show a display example of 
which two input video signals of two sources that are 
selected are letter box signals and a video signal decod- 
ed by an MPEG2 decoder is a side panel signal. 

w [0091] As shown in Figs. 12B and 12C, aljthe picture 
of each of the selected two letter box signals S42 and 
S43 of two sources that contain top and bottom non- 
picture portions is written to the picture memory. A video 
signal of the effective picture area is extracted from each 

is of the signals. In the effective picture area, the picture 
size is adjusted. The adjusted picture of each signal is 
read from the picture memory and displayed on the 
background picture. 

[0092] As shown in Fig. 12A, all the picture of a side 
20 panel signal S41 that has been decoded by the MPEG2 
decoder and that contains top and bottom non-picture 
portions is written the picture memory. 
HorizontaLSize_Va1ue (HSV) is compared with 
Display_HorizontaLSize (DHS). VerticaLSize_Value 
25 (VSV) is compared with Display_Vertial_Size (DVS). 
Corresponding to the smaller parameters, a video signal 
is extracted from the picture memory and displayed on 
the background picture. 

[0093] Thus, as shown in Fig. 12D, three pictures that 
30 are a picture G41 formed with the side panel signal S41 
decoded by the MPEG2 decoder and pictures G42 and 
G43 formed with the selected two letter box signals of 
two sources are displayed. 

[0094] Figs. 13A to 13D show a display example of 
35 which one of selected two input video signals of two 
sources is a side panel signal, the other is a letter box 
signal, and a video signal decoded by the MPEG decod- 
er is a side panel signal. 

[0095] As shown in Fig. 1 3A, all the picture of one side 
40 panel signal S51 of the selected two input video signals 
of two sources that contains left and right non-picture 
portions is written to the picture memory. A video signal 
of the effective picture area is extracted from the video 
signal written to the picture memory. In the effective pic- 
45 ture area, the picture size is adjusted. The adjusted pic- 
ture is read from the picture memory and displayed on 
the background screen. 

[0096] As shown in Fig. 13C, all the picture of the oth- 
er letter box signal S53 of the selected two input video 

so signals of two sources that contains top and bottom non- 
picture portions is written to the picture memory. A video 
signal of the effective picture area is extracted from the 
video signal written to the picture memory. In the effec- 
tive picture area, the picture size is adjusted. The ad- 

55 justed picture is read from the picture memory and dis- 
played on the background screen. 
[0097] As shown in Fig. 13B, all the picture of a letter 
box signal S52 that has been decoded by the MPEG2 
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decoder and that contains top and bottom non-picture 
portions is written to the picture memory. 
Holizontal_Size_Value (HSV) is compared with 
Display_HorizontaLSize (DHS). In addition, 
VerticaLSize_Value (VSV) is compared with 
Display_Vertical_Size (DVS). Corresponding to the 
smaller parameters, a video signal is extracted from the 
picture memory. 

[0098] Thus, as shown in Fig. 1 3D, three pictures that 
are a picture G51 formed with the side panel signal S51 
of one of the selected two input video signals of two 
sources, a picture G52 formed with the letter box signal 
S52 decoded by the MPEG2 decoder, and a picture G53 
formed with the letter box signal S53 of the other of the 
selected two input video signals of two sources are dis- 
played. 

[0099] Figs. 14A and 14B show a display example of 
which a decoded video signal is a side panel signal that 
is displayed as a reduced picture. As shown in Fig. 14A, 
all the picture of a side panel signal S61 that has been 
decoded by the MPEG2 decoder and that contains left 
and right non-picture portions is written to the picture 
memory. Horizontal_Size_Va!ue (HSV) is compared 
with DisplayJHorizontal_Size (DHS). In addition, 
Vertical_Size_Value (VSV) is compared with 
Display_Vertical_Size (DVS). Corresponding to the 
smaller parameters, a video signal is extracted from the 
picture memory. As shown in Fig. 14B, in addition to a 
picture G61 formed with the letter box signal S61, an 
on-screen picture G62 such as text is displayed. 
[0100] Figs. 15A and 15B show a display example of 
which a video signal decoded by the MPEG2 decoder 
24 is a letter box signal and displayed as a reduced pic- 
ture. As shown in Fig. 15A, all the picture of a letter box 
signal S71 that has been decoded by the MPEG2 de- 
coder 24 and that contains top and bottom non-picture 
portions is written to the picture memory. 
Horizontal_Size_Value (HSV) is compared with 
DisplayJHorizontaLSize (DHS). In addition, 
VerticaLSize_Value (VSV) is compared with 
Display_Vertial_Size (DVS). Corresponding to the 
smaller parameters, a video signal is extracted from the 
picture memory. As shown in Fig. 15B, in addition to a 
picture G71 formed with the letter box signal, an on- 
screen picture G72 such as text is displayed. 
[0101] As was described above, according to embod- 
iments of the present invention, if an input video signal 
is a side panel signal or a letter box signal, when the 
multiple-picture displaying process or the reduced pic- 
ture displaying process is performed, a signal of the ef- 
fective picture area is extracted. Thus, when the multi- 
ple-picture displaying process is performed, even if a 
side panel signal or a letter box signal is input, the pic- 
ture does not become small. In addition, when the re- 
duced picture displaying process is performed, since a 
wasteful portion is not displayed, the display screen can 
be effectively used. 

[0102] In the above-described example, in addition to 



the multiple-picture displaying process and the reduced 
picture displaying process, the picture processing por- 
tion 7 can perform a variety of processes. For example, 
the picture processing portion 7 can perform PinP (Pic- 

5 ture in Picture) process of which a picture formed with 
one of two video signals selected by the input signal se- 
lecting portion 2 is used as a parent picture, a picture 
formed with the other video signal is reduced and used 
as a child picture, and the child picture is displayed in 

10 the parent picture. In the PinP process, when only the 
effective picture area of the child picture is extracted, 
the background picture can be effectively used. 
[01 03] When an input video signal is a side panel sig- 
nal or a letter box signal, a horizontal interpolating proc- 

15 ess and a vertical interpolating process are required so 
as to display the picture on the entire screen. When an 
input video signal is a side panel signal or a letter box 
signal, the picture processing portion 7 performs a proc- 
ess for causing the picture to be displayed on the entire 

20 screen. For example, when an input video signal is an 
EDTV signal, since it is a letter box signal, a process for 
converting the number of lines is performed so as to dis- 
play the picture on the entire screen. 
[0104] According to the present invention, it is deter- 

25 mined whether or not an input video signal is a signal 
containing non-picture portions such as a side panel sig- 
nal or a letter box signal. When the multiple-picture dis- 
playing process or the reduced picture displaying proc- 
ess is performed, if an input video signal is a letter box 

30 signal or a side panel signal, only a video signal of the 
effective picture area is extracted from the video signal 
written to the picture memory and displayed. Thus, 
when the multiple-picture displaying process is per- 
formed, even if the input video signal is a side panel sig- 

35 nal or a letter box signal, the display picture does not 
become small. In addition, when the reduced picture dis- 
playing process is performed, the display area can be 
effectively used. 

[0105] Although the present invention has been 
40 shown and described with respect to a best mode em- 
bodiment thereof, it should be understood by those 
skilled in the art that the foregoing and various other 
changes, omissions, and additions in the form and detail 
thereof may be made therein without departing from the 
45 scope of the present invention . 

[0106] In so far as the embodiments of the invention 
described above are implemented, at least in part, using 
software-controlled data processing apparatus, it will be 
appreciated that a computer program providing such 
so software control and a transmission, storage or other 
medium by which such a computer program is Drovided 
are envisaged as aspects of the present invention. 



55 Claims 

1 . A picture processing apparatus, comprising: 
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determining means for determining whether or 
not an input video signal is a signal of which a 
non-picture portion is added to the periphery of 
an effective picture area; and 
picture processing means for extracting a sig- $ 
nal of the effective picture area from the input 
video signal, adjusting the picture size using 
the signal of the effective picture area, and 
combining the picture when the determined re- 
sult of said determining means represents that 10 
the input video signal is a signal of which a non- 
picture portion is added to the periphery of the 
effective picture area. 

The picture processing apparatus as set forth in 1$ 
claim 1 , 

wherein said picture processing means per- 
forms a multiple-picture displaying process for ad- 
justing the picture sizes of a plurality of input video 
signals of a plurality of sources and combining pic- 20 
tures corresponding to the plurality of input video 
signals of the plurality of sources on the background 
screen. 

The picture processing apparatus as set forth in 25 
claim 1 , 

wherein said picture processing means per- 
forms a reduced picture displaying process for re- 
ducing the picture size of the input video signal and 
combining the reduced picture on the background 30 
screen. 

The picture processing apparatus as set forth in 
claim 1 , 

wherein said determining means determines 35 
whether or not the input video signal is a signal of 
which a non-picture portion is added to the periph- 
ery of the effective picture area corresponding to in- 
formation of an interface to which the input video 
signal is input. 40 

The picture processing apparatus as set forth in 
claim 1 , 

wherein said determining means determines 
whether or not the input video signal is a signal of 
which a non-picture portion is added to the periph- 
ery of the effective picture area corresponding to in- 
formation superimposed with or added to the input 
video signal. 

50 

The picture processing apparatus as set forth in 
claim 1 , 

wherein said determining means detects a 
non signal portion of the input video signal and de- 
termines whether or not the input video signal is a 55 
signal of which a non-picture portion is added to the 
periphery of the effective picture area. 



7. The picture processing apparatus as set forth in 
claim 1 , 

wherein said determining means determines 
whether or not the input video signal is a signal of 
which a non-picture portion is added to the periph- 
ery of the effective picture area corresponding to in- 
formation contained in a transport stream that is 
transmitted. 

8. A picture processing method, comprising the steps 
of: 

(a) determining whether or not an input video 
signal is a signal of which a non-picture portion 
is added to the periphery of an effective picture 
area; and 

(b) extracting a signal of the effective picture 
area from the input video signal, adjusting the 
picture size using the signal of the effective pic- 
ture area, and combining the picture when the 
determined result at step (a) represents that the 
input video signal is a signal of which a non- 
picture portion is added to the periphery of the 
effective picture area. 

9. The picture processing method as set forth in claim 
8, 

wherein step (b) is performed by adjusting the 
picture sizes of a plurality of input video signals of 
a plurality of sources and combining pictures corre- 
sponding to the plurality of input video signals of the 
plurality of sources on the background screen. 

10. The picture processing method as set forth in claim 
8, 

wherein step (b) is performed by reducing the 
picture size of the input video signal and combining 
the reduced picture on the background screen. 

11 . The picture processing method as set forth in claim 
8, 

wherein step (a) is performed corresponding 
to information of an interface to which the input vid- 
eo signal is input. 

12. The picture processing method as set forth in claim 
8, 

wherein step (a) is performed corresponding 
to information superimposed with or added to the 
input video signal. 

13. The picture processing method as set forth in claim 
8, 

wherein step (a) is performed by detecting a 
non-signal portion of the input video signal and de- 
termining whether or not the input video signal is a 
signal of which a non-picture portion is added to the 
periphery of the effective picture area. 
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14. The picture processing method as set forth in claim 
8, 

wherein step (a) is performed corresponding 
to information contained in a transport stream that 
is transmitted. 
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